Papers with manual annotation
HGCN4MeSH: Hybrid Graph Convolution Network for MeSH Indexing (2020.acl-srw)
Copied to clipboard
| Challenge: | Recent deep learning methods for MeSH indexing fail to capture complex correlations between terms. |
| Approach: | They propose a model to learn the relationship between MeSH terms using Graph Convolution Network (GCN) they use two biGRUs to learn embedding representations of abstract and title of MeSH index text . |
| Outcome: | The proposed model is competitive with the state-of-the-art models on two datasets. |
SymbolicThought: Integrating Language Models and Symbolic Reasoning for Consistent and Interpretable Human Relationship Understanding (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing methods for constructing character relationships from plain text are time-consuming and low in coverage. |
| Approach: | They propose a human-in-the-loop framework that combines LLM-based extraction with symbolic reasoning. |
| Outcome: | The proposed framework improves annotation accuracy and consistency while significantly reducing time cost. |
Experiments with ad hoc ambiguous abbreviation expansion (D19-62)
Copied to clipboard
| Challenge: | ad hoc abbreviations are difficult to interpret for patients and nonspecialists. |
| Approach: | They propose to use morphologically annotated medical notes to expand ad hoc abbreviations without using additional domain resources. |
| Outcome: | The proposed methods outperform the previously proposed methods on Polish data but can be used for other languages. |
Low-resource Cross-lingual Event Type Detection via Distant Supervision with Minimal Effort (C18-1)
Copied to clipboard
| Challenge: | Currently, few or no language processing tools or resources exist for most languages . a problem is that there is not enough available training data even in resource-rich languages if the task is complex. |
| Approach: | They propose to use a bilingual dictionary to train machine learning in a resource-poor language . they also explore adversarial training of bilingual word representations . |
| Outcome: | The proposed approach gives similar performance in event-type detection tasks. |
Building a Part-of-Speech Tagged Corpus for Drenjongke (Bhutia) (2020.aacl-srw)
Copied to clipboard
| Challenge: | a corpus of sentences and 1379 tokens were generated for the first Drenjongke corpus . the language is considered "vulnerable," "definitely endangered" and "severely endangered." |
| Approach: | They propose to generate the first corpus of the Tibetan language using a phrase book . they propose to use 34 Part-of-Speech (PoS) tags to define the first Drenjongke corpus . |
| Outcome: | The first corpus of the Drenjongke language comprises 275 sentences and 1379 tokens . the paper plans to expand with other materials to promote further studies of the language . |
RMSSinger: Realistic-Music-Score based Singing Voice Synthesis (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for singing voice synthesis are limited to fine-grained music scores . manual adjustment destroys regularity of note durations, making fine-grain music scores "crushed" |
| Approach: | They propose a method to synthesize singing voices given realistic music scores . they use real-music-score-based Singing Voice Synthesis to generate high-quality voices . |
| Outcome: | The proposed method eliminates manual annotation and simplifies phoneme-level mel-note alignment. |
A Framework to Generate High-Quality Datapoints for Multiple Novel Intent Detection (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to detect novel intents have been tested in the last decade. |
| Approach: | They propose a framework to detect multiple novel intents with budgeted human annotation cost. |
| Outcome: | The proposed framework outperforms baseline methods in terms of accuracy and F1-score on a set of benchmark datasets. |
Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement (P18-1)
Copied to clipboard
| Challenge: | a number of post hoc explanation methods for deep neural networks have been proposed . due to the complexity of the DNNs they explain, these methods are necessarily approximations and come with their own sources of error. |
| Approach: | They propose two evaluation paradigms that cover two important classes of NLP problems . they propose LIMSSE, LRP and DeepLIFT as the most effective explanation methods . |
| Outcome: | The proposed methods are most effective for explaining deep neural networks in NLP . the proposed methods can explain complex models without manual annotation . |
Interpretable Rationale Augmented Charge Prediction System (C18-2)
Copied to clipboard
| Challenge: | Existing studies treat charge prediction as a text classification problem, but in the field of justice, every decision may be a matter of life and death. |
| Approach: | They propose to extract readable rationales from text and then create a rationale augmented classification model to enhance the prediction accuracy. |
| Outcome: | The proposed system can extract readable rationales in a high consistency with manual annotation and is comparable with the attention model in prediction accuracy. |
Towards Hierarchical Spoken Language Disfluency Modeling (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing solutions to speech dysfluency modeling are limited and expensive for low-income families. |
| Approach: | They propose a hierarchical unconstrained dysfluency modeling approach that addresses both dysfluencies transcription and detection to eliminate the need for extensive manual annotation. |
| Outcome: | The proposed approach eliminates the need for extensive manual annotation and improves the accuracy of the proposed model in phonetic transcription. |
MTO: A Multi-turn Conversational Text-to-OverpassQL Dataset for Enhanced OpenStreetMap Query Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | a framework for constructing multi-turn Text-to-OverpassQL dialogue datasets is proposed . a dataset of over 7,800 dialogues contains more than 20,000 individual utterances . |
| Approach: | They propose a framework for constructing multi-turn Text-to-OverpassQL dialogue datasets . they convert Overpass queries into syntax trees using a custom parser based on OverpassQl . |
| Outcome: | The proposed dataset includes over 7,800 dialogues, each containing 2 to 4 user utterances . it is the first multi-turn Text-to-OverpassQL dataset built upon the OverpassNL corpus . |
Building a Corpus from Handwritten Picture Postcards: Transcription, Annotation and Part-of-Speech Tagging (L18-1)
Copied to clipboard
| Challenge: | In this paper, we describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards. |
| Approach: | They describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards written in German and Swiss German. |
| Outcome: | The proposed system outperforms state-of-the-art taggers in the evaluation of the 'picture postcard corpus' containing over 11,000 handwritten postcards . |
Progressive Fine-Tuning for Cost-Effective Structured Attribute Generation in E-commerce (2026.acl-industry)
Copied to clipboard
| Challenge: | Large language models excel at structured information generation but face cost and latency challenges when deployed at scale in user-facing products. |
| Approach: | They propose a parameter efficient supervised fine-tuning pipeline for adapting a small language model to structured attribute generation in e-commerce product listing. |
| Outcome: | The proposed model reduces inference costs by 98% and latency by 70% on a large-scale product listing service while preserving an 86.4% user acceptance rate. |
CaBSALLM: Efficient Context-Aware Batch Annotation of Conversational Streams with Large Language Models (2026.acl-short)
Copied to clipboard
| Challenge: | Large-scale annotations of subjective, discourse-dependent social interactions remain a critical bottleneck in computational social science. |
| Approach: | They propose a pipeline that incorporates lightweight conversational context and a dynamic batching method to improve throughput and scalability. |
| Outcome: | The proposed pipeline improves throughput and scalability while preserving interpretive depth essential to complex social annotations. |
Toward Automatic Delegation Extraction in Japanese Law (2026.eacl-industry)
Copied to clipboard
| Challenge: | a higher-level law authorizes a lower-level to implement detailed provisions, which is called delegation. |
| Approach: | They propose a two-stage pipeline system for automatic delegation annotation in Japanese law . they extract keywords that indicate delegation using a named entity recognition approach . |
| Outcome: | The proposed system shows sufficient performance to assist manual annotation in practice. |
Thesis Proposal: Stability-Aware, Evidence-Grounded Knowledge Graph for Substance Use Disorders and Social Determinants of Health (2026.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods for NER and RE annotation are costly and difficult to scale. |
| Approach: | They propose a semantic stability framework for constructing explainable KGs using NER and RE annotations. |
| Outcome: | The proposed framework supports multi-hop reasoning, triadic SUD–SDOH–SUD mediation patterns, and feedback loop analysis. |
Guided Knowledge Generation with Language Models for Commonsense Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved notable success in commonsense reasoning tasks, benefiting from extensive world knowledge acquired through extensive pretraining. |
| Approach: | They propose a method to generate knowledge explanations and to automatically assign labels based on the probability of correct answers. |
| Outcome: | The proposed method outperforms baselines on four widely-used commonsense reasoning benchmarks and shows that it can generate high quality knowledge leading to correct answers. |
Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Adapting recipes to cultural differences presents significant importance and challenges . bridging cultural differences is a challenge, but IR can help. |
| Approach: | They propose a framework that preserves the original recipe and its cultural appropriateness for the target culture. |
| Outcome: | The proposed framework preserves the original recipe and its cultural appropriateness for the target culture while maintaining relevance to the original. |
EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI (2026.findings-acl)
Copied to clipboard
| Challenge: | Human label variation (HLV) arises when multiple labels are valid for the same instance. |
| Approach: | They propose a framework for generating and validating explanations to detect errors using large language models (LLMs) EVADE framework provides broader explanation coverage and requires less human intervention . |
| Outcome: | The proposed framework provides broader explanation coverage, requires less human intervention, and delivers better downstream performance in predicting label distributions. |
POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversion (2025.emnlp-main)
Copied to clipboard
Yuan Liu, Zhongyin Zhao, Le Tian, Haicheng Wang, Xubing Ye, Yangxiu You, Zilin Yu, Chuhan Wu, Zhou Xiao, Yang Yu, Jie Zhou
| Challenge: | Existing approaches to training document conversion models with manual annotation are costly and time-consuming, and training student models by distilling outputs from teacher models can significantly limit their performance in real-world applications. |
| Approach: | They propose a fully automated framework for constructing high-quality document extraction datasets and models capable of handling diverse document formats and layouts. |
| Outcome: | The proposed model outperforms existing models and improves on annotated documents. |
The ACoLi CoNLL Libraries: Beyond Tab-Separated Values (L18-1)
Copied to clipboard
| Challenge: | a new set of Java archives facilitates advanced manipulations of corpora annotated in TSV formats. |
| Approach: | They propose to use Java archives to facilitate advanced manipulations of corpora annotated in TSV formats. |
| Outcome: | The proposed libraries support all members of the CoNLL format family. |
ITERATE: Image-Text Enhancement, Retrieval, and Alignment for Transmodal Evolution with LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | a new framework for visual annotation of text-based questions is needed to improve performance . obtaining corresponding images through manual annotation often entails high costs . |
| Approach: | They propose a framework that uses visual modality to enhance the performance of text-based questions. |
| Outcome: | The proposed framework improves the alignment between text and images by using search engines or web scraping techniques. |
AraVQA: Building a New Arabic Factoid Visual Question Answering Dataset from Wikipedia (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Arabic VQA datasets focus on culturally-specific and dialect-aware domains. |
| Approach: | They propose a pipeline that leverages Wikipedia template tags to extract relevant information for each image and utilize it to generate a new visual question answering dataset. |
| Outcome: | The proposed pipeline can enhance existing VLMs on Arabic VQA tasks by leveraging Wikipedia template tags. |
CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Annotated data plays a critical role in training models and evaluating their performance. |
| Approach: | They propose a paradigm for Human-LLM co-annotation of unstructured texts at scale that utilizes uncertainty to estimate LLMs’ annotation capability. |
| Outcome: | The proposed model outperforms existing models on many text-annotation tasks with up to 21% performance improvement over random baseline. |
Sign Language Video Segmentation Using Temporal Boundary Identification (2025.acl-srw)
Copied to clipboard
| Challenge: | Sign language segmentation focuses on identifying temporal boundaries within video . previous methods have relied on frame-level and phrase-level segmentation. |
| Approach: | They propose to use synchronized subtitle data to facilitate temporal boundary recognition by a sequence-to-sequence model with and without attention for subtitle boundary identification. |
| Outcome: | The proposed model outperforms baseline models on optical flow data and aligned subtitles from BOBSL and YouTube-ASL. |
AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators (2024.acl-long)
Copied to clipboard
| Challenge: | generative AI is a counter-measure to misinformation, but factual claim detection suffers from inconsistency in definitions and high cost of manual annotation. |
| Approach: | They propose a framework that assists in the annotation of factual claims with the help of large language models. |
| Outcome: | The proposed framework can be used to annotate factual claims with the help of large language models and can work with or without expert supervision. |
FlexDoc: Parameterized Sampling for Diverse Multilingual Synthetic Documents for Training Document Understanding Models (2025.emnlp-industry)
Copied to clipboard
Karan Dua, Hitesh Laxmichand Patel, Puneet Mittal, Ranjeet Gupta, Amit Agarwal, Praneet Pabolu, Srikant Panda, Hansa Meghwani, Graham Horwood, Fahad Shah
| Challenge: | Document understanding models require large, diverse, and well-annotated datasets that can cost millions of dollars to collect and maintain. |
| Approach: | They propose a scalable synthetic data generation framework that combines Stochastic Schemas and Parameterized Sampling to produce realistic, multilingual semi-structured documents with rich annotations. |
| Outcome: | Experiments on key information extraction tasks show that the proposed framework improves the absolute F1 score by up to 11% while reducing annotation effort by over 90% compared to traditional hard-template methods. |
Harnessing PDF Data for Improving Japanese Large Multimodal Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Multimodal Models (LMMs) have demonstrated strong performance in English, but their effectiveness in Japanese remains limited due to the lack of high-quality training data. |
| Approach: | They propose a pipeline that leverages pretrained models to extract image-text pairs from PDFs . they use layout analysis, OCR, and vision-language pairing to enrich the training data . |
| Outcome: | The proposed pipeline extracts image-text pairs from Japanese PDFs, eliminating manual annotations. |
Making Body Movement in Sign Language Corpus Accessible for Linguists and Machines with Three-Dimensional Normalization of MediaPipe (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods of manual annotation and recognition relied on a predefinition of features and required technical knowledge. |
| Approach: | They propose a 3D normalization method for MediaPipe’s 2D pose and a novel human-readable way of representing the 3D standardized pose data. |
| Outcome: | The proposed method outperforms the common 2D normalization on a small, diverse Japanese Sign Language (JSL) sociolinguistic dataset. |
From analysis to modeling of engagement as sequences of multimodal behaviors (L18-1)
Copied to clipboard
| Challenge: | Embodied Conversational Agents (ECAs) are virtual characters that can interact with a user. |
| Approach: | They propose to endow an Embodied Conversational Agent with engagement capabilities . they use a corpus of expert-novice interactions to analyze user's engagement level . |
| Outcome: | The proposed approach analyzes user's engagement level and controls agent's behavior. |
Identifying Semantic Divergences in Parallel Text without Annotations (N18-1)
Copied to clipboard
| Challenge: | Parallel sentence pairs are sentences that are translations of each other and convey the same meaning in the source and target languages. |
| Approach: | They propose a model which detects meaning divergences in parallel sentence pairs . parallel sentence pair are translations of each other, therefore often assumed to convey the same meaning . |
| Outcome: | The proposed model detects divergences more accurately than models based on word alignments. |
From Graph to Text and Back: Semantic Fidelity in Automated Industrial Knowledge Graphs (2026.acl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often hallucinate entities or omit relations, posing unacceptable liability. |
| Approach: | They propose a self-supervised round-trip pipeline to enforce strict semantic fidelity in KG-to-text generation. |
| Outcome: | The proposed approach improves triple-extraction accuracy and verbalization faithfulness without manual annotation or massive teacher models. |
Learning from Measurements in Crowdsourcing Models: Inferring Ground Truth from Diverse Annotation Types (C18-1)
Copied to clipboard
| Challenge: | Annotated corpora are often assigned to internet workers whose judgments are reconciled by crowdsourcing models. |
| Approach: | They propose a framework for learning from rich prior knowledge to combine annotations with different structures. |
| Outcome: | The proposed model compares favorably with previous work and enables active sample selection to reduce annotation effort. |
CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understanding (2021.findings-acl)
Copied to clipboard
| Challenge: | Scientific document understanding is challenging due to the highly domain specific nature of scientific language. |
| Approach: | They propose a large, contextualized, rigorously cleaned labelled dataset for cite-worthiness detection built from extracted scientific documents. |
| Outcome: | The proposed model improves on a paragraphlevel contextualized sentence labelling model based on Longformer . the model shows a 5 F1 point improvement over SciBERT which considers only individual sentences . |
Active2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to deep learning for NLP require large amounts of labeled data. |
| Approach: | They propose an approach that iteratively selects a small number of examples for expert annotation based on their estimated utility in training the model. |
| Outcome: | The proposed approach reduces the data requirements of state-of-the-art AL strategies by 3-25% on multiple NLP tasks while achieving the same performance with virtually no additional computation overhead. |
Automatic Annotation of Grammaticality in Child-Caregiver Conversations (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing child language acquisition have been tedious and inconsistent. |
| Approach: | They propose a coding scheme for context-dependent grammaticality in child-caregiver conversations and annotate 4,000 utterances from a large corpus of transcribed conversations. |
| Outcome: | The proposed method achieves human inter-annotation agreement levels and is faster and reproducible than manual methods. |
Automatic Coding of Contingency in Child-Caregiver Conversations (2024.lrec-main)
Copied to clipboard
| Challenge: | Current research on children's language development relies on manual annotation of a small sample of children, which limits our ability to draw general conclusions about development. |
| Approach: | They propose to use automatic tools to assess contingency in children's natural interactions with caregivers by annotating a small set of data with a Transformer-based model. |
| Outcome: | The proposed model replicates existing results and generates new data-driven hypotheses. |
Improving End-to-End Bangla Speech Recognition with Semi-supervised Training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to train speech recognition systems require large annotated corpus. |
| Approach: | They propose a semi-supervised training approach that exploits large unpaired audio and text data to improve the performance of an automatic speech recognition system. |
| Outcome: | The proposed method reduces the WER of the system from 37% to 31.9%. |
A Chinese Dataset for Evaluating the Safeguards in Large Language Models (2024.findings-acl)
Copied to clipboard
Yuxia Wang, Zenan Zhai, Haonan Li, Xudong Han, Shom Lin, Zhenxuan Zhang, Angela Zhao, Preslav Nakov, Timothy Baldwin
| Challenge: | a recent study has shown that large language models can produce harmful responses, exposing users to unexpected risks. |
| Approach: | They propose a dataset for the safety evaluation of Chinese LLMs in Mandarin Chinese . they extend the dataset to better identify false negative and false positive examples . |
| Outcome: | The proposed dataset is for the safety evaluation of Chinese LLMs, and is based on a Chinese dataset. |
Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent work shows that Relation Extraction tasks can be recasted as Textual Entailment tasks using verbalizations. |
| Approach: | They propose to recasted RE tasks as Textual Entailment tasks using verbalizations . they show that entailment reduces the need for manual annotation to 50% and 20% . |
| Outcome: | The proposed method reduces the need for manual annotation to 50% and 20% in event argument extraction tasks while achieving the same performance as with full training. |
Learning Sentiment Composition from Sentiment Lexicons (C18-1)
Copied to clipboard
Orith Toledo-Ronen, Roy Bar-Haim, Alon Halfon, Charles Jochim, Amir Menczel, Ranit Aharonov, Noam Slonim
| Challenge: | Sentiment composition is a fundamental problem in sentiment analysis. |
| Approach: | They propose a method for learning sentiment composition from a large, unlabeled corpus . they automatically generate large sentiment lexicons of bigrams and unigrams . |
| Outcome: | The proposed approach is validated through manual annotation and sentiment classification experiments with phrase-level and sentence-level benchmarks. |
Towards Interactive Annotation for Hesitation in Conversational Speech (2020.lrec-1)
Copied to clipboard
| Challenge: | recognizing affects in spontaneous, non acted speech is expensive in both human resources and time. |
| Approach: | They aim to automatize the labeling of hesitant speech as a marker of expressed uncertainty. |
| Outcome: | The proposed method shows that the number of filled pauses and vowel duration increases with the degree of hesitation, and that automatic prediction of the hesitation degree reaches encouraging RMSE results of 1.6. |
An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for constructing process supervision training data are costly or suffer from poor quality. |
| Approach: | They propose a framework called EpicPRM which annotates each intermediate reasoning step based on its quantified contribution and uses an adaptive binary search algorithm to enhance annotation precision and efficiency. |
| Outcome: | The proposed framework improves annotation precision and efficiency and can be used to train a high-quality training dataset with 50k annotated intermediate steps. |
Advancing Large Language Model Attribution through Self-Improving (2024.emnlp-main)
Copied to clipboard
Lei Huang, Xiaocheng Feng, Weitao Ma, Liang Zhao, Yuchun Fan, Weihong Zhong, Dongliang Xu, Qing Yang, Hongtao Liu, Bing Qin
| Challenge: | Teaching large language models to generate text with citations to evidence sources requires high-quality attribution data, which is costly and labor-intensive. |
| Approach: | They propose a framework for iteratively improving the attribution capability of large language models (LLMs) by attributing output to verifiable sources. |
| Outcome: | Experiments on three open-domain question-answering datasets show that START improves in aggregating information across multiple sources. |
Omni-Chart-600K: A Comprehensive Dataset of Chart Types for Chart Understanding (2025.findings-naacl)
Copied to clipboard
Shulei Wang, Shuai Yang, Wang Lin, Zirun Guo, Sihang Cai, Hai Huang, Ye Wang, Jingyuan Chen, Tao Jin
| Challenge: | Existing chart-related training methods lack capabilities in information extraction, mathematical reasoning, and understanding of multiple chart types. |
| Approach: | They propose a two-stage training strategy and method for jointly training a vision encoder tailored for multi-type charts to address the deficiencies in chart types and limited scope of chart tasks in existing datasets. |
| Outcome: | The proposed dataset includes 21 diverse chart types and tasks, including data retrieval and mathematical reasoning. |
Practical, Efficient, and Customizable Active Learning for Named Entity Recognition in the Digital Humanities (N19-1)
Copied to clipboard
Alexander Erdmann, David Joseph Wrisley, Benjamin Allen, Christopher Brown, Sophie Cohen-Bodénès, Micha Elsner, Yukun Feng, Brian Joseph, Béatrice Joyeux-Prunel, Marie-Catherine de Marneffe
| Challenge: | Scholars in interdisciplinary fields like the Digital Humanities are increasingly interested in semantic annotation of specialized corpora. |
| Approach: | They propose an active learning solution for named entity recognition that maximizes a custom model’s improvement per additional unit of manual annotation. |
| Outcome: | The proposed model reduces required annotation by 20-60% and outperforms a competitive active learning baseline. |
CXR Data Annotation and Classification with Pre-trained Language Models (2022.coling-1)
Copied to clipboard
Nina Zhou, Ai Ti Aw, Zhuo Han Liu, Cher heng Tan, Yonghan Ting, Wen Xiang Chen, Jordan sim zheng Ting
| Challenge: | Existing tools for clinical data annotation are limited to specific institutions due to differences in writing style, structure, language use and label definition. |
| Approach: | They propose a weak supervision annotation framework with two improvements over existing ones . the framework provides an efficient form of sample selection and data auto-annotation . |
| Outcome: | The proposed framework provides better results for clinical data annotation tasks compared to existing frameworks. |
Classifying Sluice Occurrences in Dialogue (L18-1)
Copied to clipboard
| Challenge: | Ellipsis is an important challenge for natural language processing systems, says a new paper . previous work on ellipsis focused on news data, but sluicing presents a challenge for dialogue systems . |
| Approach: | They describe a corpus of 4100 sluice occurrences from the NYTimes Gigaword corpus . they build a classifier model to automatically classify slujce . |
| Outcome: | The proposed corpus contains 4100 sluice occurrences, with an accuracy of 67% . the work will support empirical research into slujcing in dialogue systems . |
Conditional Causal Relationships between Emotions and Causes in Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the causal relationships between emotions and causes focus on extracting causally related clauses from documents, but none considers whether context clauses are indispensable for extracted clauses to be causally linked. |
| Approach: | They propose a task to determine whether an input pair of emotion and cause has a valid causal relationship under different contexts. |
| Outcome: | The proposed task identifies whether an input pair of emotion and cause has a valid causal relationship under different contexts and then fine-tunes the prediction results based on the characteristics of the input clauses. |
PST 2.0 – Corpus of Polish Spatial Texts (2020.lrec-1)
Copied to clipboard
| Challenge: | In this paper, we focus on modeling spatial expressions in texts. |
| Approach: | They propose guidelines for annotating the PST 2.0 corpus of Polish Spatial Texts based on existing standards for English and discuss modifications to the guidelines to the characteristics of the language. |
| Outcome: | The proposed framework is based on three existing standards for English and ISO-Space1.4 from SpaceEval 2014 . |
Creation and evaluation of timelines for longitudinal user posts (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for segmenting user posts into timelines improve quality and cost of manual annotation. |
| Approach: | They propose a set of methods for segmenting longitudinal user posts into timelines likely to contain interesting moments of change in a user’s behaviour based on their online posting activity. |
| Outcome: | The proposed framework is able to evaluate two different social media datasets and compares with existing models. |
Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations (2024.acl-long)
Copied to clipboard
| Challenge: | Vision-and-language models with separate encoders for each modality are limited in availability. |
| Approach: | They propose a multilingual benchmark that offers (partial) translations of ImageNet labels to 100 languages, built without machine translation or manual annotation. |
| Outcome: | The proposed model outperforms models on English and low-resource languages. |
CHICA: A Developmental Corpus of Child-Caregiver’s Face-to-face vs. Video Call Conversations in Middle Childhood (2024.lrec-main)
Copied to clipboard
Dhia Elhak Goumri, Abhishek Agrawal, Mitja Nikolaus, Hong Duc Thang Vu, Kübra Bodur, Elias Emmar, Cassandre Armand, Chiara Mazzocconi, Shreejata Gupta, Laurent Prévot, Benoit Favre, Leonor Becerra-Bonache, Abdellah Fourtassi
| Challenge: | Existing studies of language-in-interaction focus on the two ends of the developmental spectrum, i.e., early childhood and adulthood, leaving a gap in our knowledge about how development unfolds, especially across middle childhood. |
| Approach: | They propose to use CHICA to analyze child-caregiver conversations at home . they use mobile, lightweight eye-tracking and head motion detection to optimize the naturalness of the recordings. |
| Outcome: | The proposed corpus of child-caregiver conversations at home was compared with a previous corpus based on a set of conversations between children aged 7, 9, and 11 years old. |
Quality and Efficiency of Manual Annotation: Pre-annotation Bias (2022.lrec-1)
Copied to clipboard
| Challenge: | Annotators using pre-annotation are less efficient at producing high quality annotations. |
| Approach: | They propose to use an automatic pre-annotation for a task to judge annotation quality . they also evaluate the effect of automatic linguistically-based checks on the same data . |
| Outcome: | The proposed method improves the quality of annotated sentences without reducing quality. |
McQueen: a Benchmark for Multimodal Conversational Query Rewrite (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on conversational-related tasks that involve drawing information from more than one modality. |
| Approach: | They propose a task of multimodal conversational query rewrite which performs query . they collect a large-scale visual conversation dataset and benchmark it against other tasks . |
| Outcome: | The proposed task performs on a large-scale visual conversation dataset . it eliminates coreference and ellipsis in the original query without changing its semantic information. |
A Web-based System for Crowd-in-the-Loop Dependency Treebanking (L18-1)
Copied to clipboard
| Challenge: | Existing treebanks are limited in size, genre, and topic coverage, making manual annotation time-consuming and expensive. |
| Approach: | They propose a web-based interactive tool for editing dependency trees that uses machine learning to accelerate annotation. |
| Outcome: | CROWDTREE is a web-based interactive tool for editing dependency trees . it can train a parsing model during the annotation process and can even be compatible with Mechanical Turk. |
Toeing the Party Line: Election Manifestos as a Key to Understand Political Discourse on Twitter (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on political positioning on Twitter has tended to focus on manifestos rather than social media since it is ambiguous and dependent on social context. |
| Approach: | They propose to use hashtags as a signal to fine-tune text representations for politicians' tweets using a hashtag-based method to predict pairwise positional similarities between parties from the manifesto case to the Twitter case. |
| Outcome: | The proposed method matches politicians' statements to official lines of the parties' tweets, even when only small subsets from shorter time periods are available. |
The UD-NewsCrawl Treebank: Reflections and Challenges from a Large-scale Tagalog Syntactic Annotation Project (2025.acl-long)
Copied to clipboard
| Challenge: | UD-NewsCrawl is the largest Tagalog treebank to date, with 15.6k trees manually annotated according to the Universal Dependencies framework. |
| Approach: | They propose to use UD-NewsCrawl to annotate Tagalog trees using the Universal Dependencies framework. |
| Outcome: | The proposed treebanks are based on the Universal Dependencies framework and have 15.6k trees annotated manually. |
Generating Questions, Answers, and Distractors for Videos: Exploring Semantic Uncertainty of Object Motions (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing video QADs are generated using video captions, incurring significant costs and resulting in bias. |
| Approach: | They propose to use temporal motion to describe video objects to generate diverse QADs focusing on different objects and interactions. |
| Outcome: | The proposed approach improves consistency and diversity of generated QADs on the NExT-QA and Perception Test benchmarks. |
Uchaguzi-2022: A Dataset of Citizen Reports on the 2022 Kenyan Election (2025.coling-main)
Copied to clipboard
Roberto Mondini, Neema Kotonya, Robert L Logan IV, Elizabeth M. Olson, Angela Oduor Lungati, Daniel Odongo, Tim Ombasa, Hemank Lamba, Aoife Cahill, Joel Tetreault, Alejandro Jaimes
| Challenge: | Systematically organizing and geotagging large amounts of crowdsourced information requires substantial manual effort, often led by volunteers. |
| Approach: | They present a dataset of 14k citizen reports related to the 2022 Kenyan General Election . they investigate whether language models can assist in scalably categorizing and geotagging reports . |
| Outcome: | The proposed dataset aims to show whether language models can assist in categorizing and geotagging reports, thus highlighting its potential application in the AI for Social Good space. |
A Dataset of Translational Equivalents Built on the Basis of plWordNet-Princeton WordNet Synset Mapping (2020.lrec-1)
Copied to clipboard
| Challenge: | a dataset of 11,000 Polish-English translational equivalents is presented . the dataset is a novum in the wordnet domain and can facilitate the precision of bilingual NLP tasks. |
| Approach: | They present a dataset of Polish-English translational equivalents linked by three types of equivalence links. |
| Outcome: | The proposed dataset contains 11,000 Polish-English translational equivalents . the resulting subsets are based on a manual annotation process and a set of formal features . |
SCALE: Towards Collaborative Content Analysis in Social Science with Large Language Model Agents and Human Intervention (2025.acl-long)
Copied to clipboard
| Challenge: | Content analysis is labor-intensive and time-consuming process that requires multiple rounds of manual annotation, domain expert discussion, and rule-based refinement. |
| Approach: | They propose a multi-agent framework that effectively Simulates Content Analysis via Large language model (LLM) ag Ents. |
| Outcome: | The proposed framework achieves human-approximated performance across various content analysis tasks. |
SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages (2025.coling-main)
Copied to clipboard
| Challenge: | Question Answering datasets are scarce for languages other than English due to the cost and difficulties of collection and manual annotation. |
| Approach: | They propose a method for generating and validating QA datasets for low-resource languages . they use English data as context to generate synthetic multiple-choice (MC) question-answer pairs . |
| Outcome: | The proposed method maintains quality, reduces likelihood of factual errors, and circumvents costly annotation. |
Introducing a New Dataset for Event Detection in Cybersecurity Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | a large amount of text data is produced to report and discuss cyber vulnerabilities . detecting cybersecurity events is necessary to keep us informed about the fast growing number of such events reported in text. |
| Approach: | They propose a dataset characterizing the manual annotation for 30 important cybersecurity event types and a large dataset to develop deep learning models. |
| Outcome: | The proposed dataset characterizes the manual annotation for 30 important event types and supports the modeling of document-level information to improve the performance. |
Comparative evaluation of boundary-relaxed annotation for Entity Linking performance (2023.acl-long)
Copied to clipboard
| Challenge: | Entity Linking is a critical step for information extraction, allowing the retrieval and understanding of information from unstructured textual sources. |
| Approach: | They propose to use noisy datasets to generate noisy versions of annotated entity mentions and then train three Entity Linking models on this data. |
| Outcome: | The proposed model can be used to associate NE mentions to a single concept in an ontology, allowing for better indexing and relation extraction. |
Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs (2024.findings-emnlp)
Copied to clipboard
Junjie Wang, Mingyang Chen, Binbin Hu, Dan Yang, Ziqi Liu, Yue Shen, Peng Wei, Zhiqiang Zhang, Jinjie Gu, Jun Zhou, Jeff Pan, Wen Zhang, Huajun Chen
| Challenge: | Recent studies have attempted to enhance the performance of large language models (LLMs) in complex question-answering (QA) tasks by combining step-wise planning with external retrieval. |
| Approach: | They propose a framework for enhancing LLMs’ planning capabilities by using planning data derived from knowledge graphs (KGs). |
| Outcome: | The proposed framework improves LLMs’ planning capabilities by using knowledge graphs (KGs) the proposed framework is compared with existing frameworks on multiple datasets and shows that it is effective for large language models. |
What Makes Reading Comprehension Questions Difficult? (2022.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that natural language understanding benchmarks are not able to measure future progress . a crowdsourcing approach is needed to collect diverse examples without sacrificing diversity or coverage. |
| Approach: | They crowdsource multiple-choice reading comprehension questions for passages from seven sources . they find passage source, length, and readability measures do not significantly affect question difficulty . |
| Outcome: | The results show that passage source, length, and readability measures do not significantly affect question difficulty. |
Leveraging Medical Literature for Section Prediction in Electronic Health Records (D19-1)
Copied to clipboard
| Challenge: | Prior approaches to section prediction have only used text data from EHRs and required significant manual annotation. |
| Approach: | They propose to use sections from medical literature to train models to predict sections in EHRs. |
| Outcome: | The proposed model uses sections from medical literature that contain similar content to those found in EHR sections. |
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for process-oriented math reward models rely on manual annotation. |
| Approach: | They propose a process-oriented math process reward model called Math-shepherd which assigns a reward score to each step of math problem solutions. |
| Outcome: | The proposed model breaks the bottleneck of manual supervision in two scenarios. |
A Domain-Specific Dataset of Difficulty Ratings for German Noun Compounds in the Domains DIY, Cooking and Automotive (2020.lrec-1)
Copied to clipboard
| Challenge: | a dataset with difficulty ratings for 1,030 closed noun compounds is presented . authors use a simple compound splitter to identify compound types in domain-specific texts . |
| Approach: | They present a German closed noun compound dataset with difficulty ratings . they used a simple compound splitter to identify compounds in texts . |
| Outcome: | The proposed dataset has difficulty ratings for 1,030 closed noun compounds extracted from domain-specific texts for do-it-ourself, cooking and automotive. |
CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesis (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Cultural competence is defined as the ability to understand and adapt to multicultural contexts. |
| Approach: | They propose a framework that uses a hierarchical multilingual taxonomy and a Retrieval-Augmented Generation to synthesize culturally relevant question-answer pairs. |
| Outcome: | The proposed framework contains a hierarchical multilingual taxonomy covering 12 primary and 130 secondary topics and a Retrieval-Augmented Generation (RAG)-based methodology leveraging factual knowledge to synthesize culturally relevant question-answer pairs. |
Named Entity Recognition in Estonian 19th Century Parish Court Records (2022.lrec-1)
Copied to clipboard
| Challenge: | a corpus of 19th century Parish Court records annotated for named entities (NE) in Estonian is a valuable resource for historians, linguists and the public at large. |
| Approach: | They propose to annotate a corpus of Estonian Parish Court records annotated for named entities (NE) and report on named entity recognition experiments using this corpus. |
| Outcome: | The proposed model achieves microaverage F1 score of 93.6, comparable to state-of-the-art NER performance on the contemporary Estonian. |
All That Glitters is Not Gold: Improving Robust Retrieval-Augmented Language Models with Fact-Centric Preference Alignment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn adaptive retrieval for noisy documents lack prior filtering and may lead to the loss of crucial information. |
| Approach: | They propose a method to improve retrieval performance without prior filtering . they use LLMs self-generated synthetic data as training data without manual annotation . |
| Outcome: | The proposed method performs positive document mining based on factual consistency and uses LLMs self-generated synthetic data as training data without manual annotation. |
A Herd of Language Models Makes a Better Zero-shot Annotator for Clinical Named Entity Recognition (2026.findings-acl)
Copied to clipboard
| Challenge: | Clinical named entity recognition (NER) is a core task in clinical NLP. |
| Approach: | They propose a label-modeling method for M**ulti-LLM **A**nnotation using **R**epresentation learning to capture contextual similarity. |
| Outcome: | The proposed method improves the average F1 score by 8.6% over zero-shot baselines while reducing annotation costs. |
Active Learning for Natural Language Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to NLG are limited by the lack of annotated data. |
| Approach: | They propose to use active learning to reduce the cost of manual annotation to improve annotation efficiency by selecting the most informative examples to label. |
| Outcome: | The proposed approach surpasses baseline of random example selection in some cases but not in others. |
ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models (2024.emnlp-main)
Copied to clipboard
Yuxiang Zhang, Jing Chen, Junjie Wang, Yaxin Liu, Cheng Yang, Chufan Shi, Xinyu Zhu, Zihao Lin, Hanwen Wan, Yujiu Yang, Tetsuya Sakai, Tian Feng, Hayato Yamana
| Challenge: | Currently, tool-augmented large language models (LLMs) only achieve total scores of 45.3 and 37.0, respectively, on a scale of 100. |
| Approach: | They propose a multi-level diagnostic process to assess the LLM's hallucinations through two perspectives: depth and breadth. |
| Outcome: | The proposed diagnostic process assesses the hallucinations of large language models through two perspectives: depth and breadth. |
Finding Spoken Identifications: Using GPT-4 Annotation for an Efficient and Fast Dataset Creation Pipeline (2024.lrec-main)
Copied to clipboard
Maliha Jahan, Helin Wang, Thomas Thebaud, Yinglun Sun, Giang Ha Le, Zsuzsanna Fagyal, Odette Scharenborg, Mark Hasegawa-Johnson, Laureano Moro Velazquez, Najim Dehak
| Challenge: | Existing datasets that are limited to a few dialects, ethnicities, and age groups are not annotated considering these factors. |
| Approach: | They propose a semi-automated dataset creation pipeline that leverages large language models to perform two complex annotation tasks using human annotations as ground truths. |
| Outcome: | The proposed pipeline reduces time required for the filtering and tagging tasks while losing no important information. |
CaRB: A Crowdsourced Benchmark for Open IE (D19-1)
Copied to clipboard
| Challenge: | Open Information Extraction (Open IE) systems have been evaluated traditionally via manual annotation. |
| Approach: | They propose to use a dataset to score Open IE systems by matching system predictions with benchmark datasets. |
| Outcome: | The proposed framework matches predictions with the benchmark dataset and is noisy and inconsistent. |
MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Metadata extraction relies heavily on manual annotation of documents. |
| Approach: | They propose a framework that leverages Large Language Models to automatically extract metadata attributes from scientific papers covering datasets of languages other than Arabic. |
| Outcome: | The proposed framework automates the extraction of metadata attributes from Arabic scientific papers using large language models. |
MONETA: Multimodal Industry Classification through Geographic Information with Multi Agent Systems (2026.acl-long)
Copied to clipboard
| Challenge: | MONETA is the first multimodal industry classification benchmark with text and geospatial sources. |
| Approach: | They propose a multimodal industry classification benchmark using text and geospatial sources. |
| Outcome: | The proposed model increases the accuracy of the existing models by 22.80%. |
“A Tale of Two Movements’: Identifying and Comparing Perspectives in #BlackLivesMatter and #BlueLivesMatter Movements-related Tweets using Weakly Supervised Graph-based Structured Prediction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a weakly supervised graph-based approach to model #BLM-related tweets is difficult to obtain . |
| Approach: | They propose a weakly supervised graph-based approach that explicitly models perspectives in #BackLivesMatter-related tweets. |
| Outcome: | The proposed model outperforms multitask baselines by a large margin. |
Figure Me Out: A Gold Standard Dataset for Metaphor Interpretation (2020.lrec-1)
Copied to clipboard
| Challenge: | Metaphor comprehension and understanding is a complex cognitive task that requires interpreting metaphors by grasping the interaction between the meaning of their target and source concepts. |
| Approach: | They propose an automatic retrieval approach to annotate verb-noun metaphors in text . they validated their approach by annotating around 1,500 metaphors from tweets . |
| Outcome: | The proposed method reduces the workload on annotators and maintains consistency . it can be used to interpret verb-noun metaphoric expressions in tweets . |
KnowCoder-X: Boosting Multilingual Information Extraction via Code (2025.findings-acl)
Copied to clipboard
Yuxin Zuo, Wenxuan Jiang, Wenxuan Liu, Zixuan Li, Long Bai, Hanbin Wang, Yutao Zeng, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng
| Challenge: | Empirical evidence indicates that Large Language Models exhibit spontaneous cross-lingual alignment in Information Extraction (IE) however, a significant imbalance across languages persists, highlighting an underlying deficiency. |
| Approach: | They propose a code LLM with advanced cross-lingual and multilingual capabilities for universal IE that standardizes the representation of multilingual schemas using Python classes and conducts IE alignment instruction tuning on translated instance prediction task. |
| Outcome: | The proposed model surpasses ChatGPT and SoTA by 30.17% without training in 29 unseen languages and significantly improves cross-lingual IE transferability. |
Exploring LLM Annotation for Adaptation of Clinical Information Extraction Models under Data-sharing Restrictions (2025.findings-acl)
Copied to clipboard
| Challenge: | In-hospital text data often contains valuable clinical information, yet fine-tuned small language models (SLMs) for information extraction remain challenging due to differences in formatting and vocabulary across institutions. |
| Approach: | They leverage large language models to annotate the target domain data for adaptation . they use in-hospital text data to extract clinical information . |
| Outcome: | The proposed model outperforms manual annotation on four clinical information extraction tasks with a larger number of annotated data. |
Benchmarking Query-Conditioned Natural Language Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Query-conditioned natural language inference (QC-NLI) is a new approach to detect inconsistencies in large language models. |
| Approach: | They propose a task of Query-Conditioned Natural Language Inference to determine the semantic relationship between two documents conditioned on a query. |
| Outcome: | The proposed task is based on a query-conditioned natural language inference (QC-NLI) it is used to determine the relationship between the premise and hypothesis given a given query. |
A Corpus of German Reddit Exchanges (GeRedE) (2020.lrec-1)
Copied to clipboard
| Challenge: | Reddit is a popular online platform combining social news aggregation, discussion and microblogging. |
| Approach: | They propose a method to filter out German data and further pre-processing steps to find out what is linguistically peculiar in the German data. |
| Outcome: | The proposed method filters out German data and includes metadata and annotation layers. |
PAUSE: Positive and Annealed Unlabeled Sentence Embedding (2021.emnlp-main)
Copied to clipboard
Lele Cao, Emil Larsson, Vilhelm von Ehrenheim, Dhiana Deva Cavalcanti Rocha, Anna Martin, Sonja Horn
| Challenge: | Sentence embedding is a set of effective and versatile techniques for converting raw text into numerical vector representations. |
| Approach: | They propose a generic and end-to-end approach to embed sentences from a partially labeled dataset using supervised methods. |
| Outcome: | The proposed approach achieves state-of-the-art results using only a small fraction of labeled sentence pairs on various benchmark tasks. |
Pedagogical Alignment of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are often used without pedagogical fine-tuning and provide immediate answers rather than guiding students through the problem-solving process. |
| Approach: | They propose a method for constructing large-scale preference datasets using synthetic data generation techniques that eliminates the need for manual annotation. |
| Outcome: | The proposed methods outperform standard supervised fine-tuning (SFT) and improve alignment accuracy by 13.1% and 8.7% respectively. |
ClinAlign: Scaling Healthcare Alignment from Clinician Preference (2026.findings-acl)
Copied to clipboard
Shiwei Lyu, Xidong Wang, Hao Zhu, Lei Liu, Chaohe Zhang, Jian Wang, Jinjie Gu, Benyou Wang, Yue Shen
| Challenge: | Existing methods for aligning open-ended outputs with fine-grained clinician preferences are weakly grounded in professional guidelines. |
| Approach: | They propose a framework to align large language models' outputs with fine-grained clinician preferences . they propose 119 broadly reusable, clinically grounded principles organized by clinical dimensions . |
| Outcome: | The proposed framework outperforms existing models on HealthBench-Hard and Deepseek-R1 and o3. |
Invisible to People but not to Machines: Evaluation of Style-aware HeadlineGeneration in Absence of Reliable Human Judgment (2020.lrec-1)
Copied to clipboard
| Challenge: | Using a data alignment strategy and different training/testing settings, we aim at decoupling content from style and preserving the latter in generation. |
| Approach: | They propose a fine-grained evaluation strategy based on automatic classification to evaluate generated headlines' quality in terms of their newspaper-compliance. |
| Outcome: | The proposed model learns newspaper-specific style, but humans aren't reliable judges for this task, and deserves particular care in its design. |
Unlocking Legal Knowledge: A Multilingual Dataset for Judicial Summarization in Switzerland (2025.findings-emnlp)
Copied to clipboard
Luca Rolshoven, Vishvaksenan Rasiah, Srinanda Brügger Bose, Sarah Hostettler, Lara Burkhalter, Matthias Stürmer, Joel Niklaus
| Challenge: | a dataset of 20K rulings from the Swiss Federal Supreme Court is lacking in legal headnotes due to the high cost of manual annotation. |
| Approach: | They propose a dataset that contains 20K rulings from the Swiss Federal Supreme Court . they fine-tune open models and compare them to larger general-purpose and reasoning-tunned LLMs . |
| Outcome: | The proposed dataset contains 20K rulings from the Swiss Federal Supreme Court with headnotes in German, French, and Italian. |
Morpheme Sense Disambiguation: A New Task Aiming for Understanding the Language at Character Level (2024.lrec-main)
Copied to clipboard
| Challenge: | Morphemes are a strong linguistic feature to capture lexical semantics, but lack of morpheme-informed resources and the expense of manual annotations hinder morphme-enhanced methods. |
| Approach: | They propose a task of Morpheme Sense Disambiguation with two subtasks in-text and in-word to generalize morpheme features on more tasks. |
| Outcome: | The proposed tasks are based on two morpheme-annotated datasets for Chinese . the best model yields a promising precision of 77.66% on in-text and 88.19% on in word . |
BNLP: A Text Annotation Platform for Quality Control of LLM-Generated Annotations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing annotation tools lack support for Large Language Models (LLMs) or use LLMs as one-off preannotation engines, compromising data reliability. |
| Approach: | They propose a text annotation platform that embeds LLM-assisted labeling into a quality-aware collaborative workflow. |
| Outcome: | Experiments show that BNLP reduces annotation time by 74.3% and improves annotation quality by 11.6% over purely manual annotation in LLM-assisted settings. |
From Noise to Clarity: Filtering Real and LLM-Generated Samples for Enhanced Intent Detection (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using large language models for generating synthetic samples for data augmentation can cause problems with the generalization ability of classification models. |
| Approach: | They propose an interpretable Sample Filter by Topic Modeling framework that allows for filtering by topic and a 'sampler by topic' framework. |
| Outcome: | The proposed framework reduces the quantity of real and synthetic samples while improving the performance of the classification models. |
Scalable and Culturally Specific Stereotype Dataset Construction via Human-LLM Collaboration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for detecting and mitigating embedded stereotypes rely on carefully annotated datasets like StereoSet and CrowS-Pairs, which are only in English and reflect stereotypes from a few English-speaking countries. Existing datasets, especially translation-based ones, often overlook such cultural distinctions. |
| Approach: | They propose a cost-efficient human-LLM collaborative annotation framework to construct a Spanish-language stereotype dataset spanning multiple Spanish-speaking countries. |
| Outcome: | The proposed framework can identify nuanced, region-specific biases across Spanish-supporting LLMs and is adaptable to other languages and regions. |
MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used globally across many languages, but their English-centric pretraining raises concerns about cross-lingual disparities for cultural awareness . |
| Approach: | They introduce an automatic multilingual framework for evaluating cultural awareness in large language models across languages, regions, and topics. |
| Outcome: | The framework evaluates open-ended text generation, capturing how models express culturally grounded knowledge in natural language. |
Segmentation of Complex Question Turns for Argument Mining: A Corpus-based Study in the Financial Domain (2024.lrec-main)
Copied to clipboard
| Challenge: | Earnings Conference Calls (ECCs) are a favoured domain for the study of argumentation in context and the extraction of Argumentative Discourse Units (ADUs). |
| Approach: | Earnings Conference Calls (ECCs) are favoured domain for study of argumentation in context and extraction of Argumentative Discourse Units (ADUs). |
| Outcome: | ECCs are favoured for study of argumentation in context and extraction of Argumentative Discourse Units (ADUs). |
Sequence-to-Sequence Language Models for Character and Emotion Detection in Dream Narratives (2024.lrec-main)
Copied to clipboard
| Challenge: | Sigmund Freud's interpretation of dreams has been central to understanding human consciousness for centuries. |
| Approach: | They propose to automate the annotation process by using a natural language framework . they evaluate the impact of model size, prediction order of characters, and consideration of proper names and character traits . |
| Outcome: | The proposed model performs better with a large language model while having 28 times fewer parameters. |
HopWeaver: Cross-Document Synthesis of High-Quality and Authentic Multi-Hop Questions (2026.acl-long)
Copied to clipboard
| Challenge: | Multi-Hop Question Answering (MHQA) is a critical benchmark for evaluating the model’s ability to integrate information from diverse sources. |
| Approach: | They propose a framework that synthesizes authentic multi-hop questions without manual annotation without the need for manual guidance. |
| Outcome: | The proposed framework synthesizes bridge and comparison questions without human intervention and achieves comparable or superior quality to human-annotated datasets at a lower cost. |
Spanish Resource Grammar Version 2023 (2024.lrec-main)
Copied to clipboard
| Challenge: | Using the Freeling morphological analyzer, we encode a strict notion of grammaticality in the Spanish resource grammar. |
| Approach: | They propose to use the HPSG formalism to encode a Spanish resource grammar with a manually verified treebank of 2,291 sentences. |
| Outcome: | The proposed grammars encode a complex set of hypotheses about syntax and a strict notion of grammaticality making them a resource for natural language processing applications in computer-assisted language learning. |
Search-in-Context: Efficient Multi-Hop QA over Long Contexts via Monte Carlo Tree Search with Dynamic KV Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multihop question answering (MHQA) over long contexts are often neglecting explicit reasoning or incurring expensive computational costs due to full-attention mechanisms over long contextuals. |
| Approach: | They propose a framework that integrates Monte Carlo Tree Search (MCTS) with dynamic key-value retrieval to enable iterative, context-aware reasoning. |
| Outcome: | The proposed framework integrates Monte Carlo Tree Search (MCTS) with dynamic key-value (KV) retrieval to enable iterative, context-aware reasoning. |
A Survey on Efficient Large Language Model Training: From Data-centric Perspectives (2025.acl-long)
Copied to clipboard
Junyu Luo, Bohan Wu, Xiao Luo, Zhiping Xiao, Yiqiao Jin, Rong-Cheng Tu, Nan Yin, Yifan Wang, Jingyang Yuan, Wei Ju, Ming Zhang
| Challenge: | achieving data-efficient post-training of Large Language Models is a key research question. |
| Approach: | They propose a taxonomy of data-efficient LLM post-training methods from a data-centric perspective. |
| Outcome: | The proposed methods cover data selection, data quality enhancement, synthetic data generation, data distillation and compression, and self-evolving data ecosystems. |
Multimodal Large Language Models for Multi-Subject In-Context Image Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text-to-image (T2I) generation have enabled visually coherent image synthesis from descriptions, but generating images containing multiple given subjects remains challenging. |
| Approach: | They propose a model that enables automatic and scalable data generation without manual annotations to overcome the data scarcity. |
| Outcome: | The proposed model overcomes the data scarcity and lacks manual annotations. |